Advertising methods for advertising time slots and embedded objects

ABSTRACT

Disclosed is a process for an advertising bidding system in which advertisers can bid on advertising time slots or enhanced ads that can be inserted in either prerecorded or live television programming. A high degree of flexibility is provided by allowing embedded ads to be purchased and displayed in a very short time period. This allows TV broadcasters to increase revenues and allows advertisers to have more flexibility in advertising. Objects can be inserted in the display video broadcast using enhancements such as by using mark-up language overlay pages or by video combination techniques. The effect of these ads can be evaluated for advertising campaigns.

RELATED APPLICATIONS

This application is a Continuation of U.S. application Ser. No.10/745,899, filed on Dec. 23, 2003, which is incorporated herein byreference in its entirety.

BACKGROUND OF THE INVENTION

a. Field of the Invention

The present invention pertains generally to TV advertising and morespecifically to methods of selling TV advertising using an auctioningmethod.

b. Description of the Background

TV advertising has typically been sold in time slots, such as 30-secondtime slots or 1-minute time slots that are scheduled during breaks inthe TV programming. These breaks may be prescheduled at certain timesduring prerecorded programs or may be scheduled instantaneously, atvarious times, during live programming such as sporting events.

Scheduled advertising is budgeted and paid for well in advance of theairing of the television programs. For example, advertising slots aretypically assigned months in advance of the airing of the program. Thisis true for both live and prerecorded programming.

Such long required lead times limit the flexibility of advertisers andbroadcasters for various changing conditions such as budget changes,changes in the popularity or interest in programs and other suchfactors. As a result, advertisers may be required to purchaseadvertising spots for programs that later turn out to be unpopular, orbe limited in the amount of advertising spots that they can purchase forprograms that later turn out to be popular. Additionally, interest maychange even during the airing of a program. For example, lopsided orvery close scores during a sporting event may cause a change in publicinterest. Both advertisers and television broadcasters suffer fromreduced revenues as a result of these disadvantages.

SUMMARY OF THE INVENTION

The present invention overcomes the disadvantages and limitations of theprior art by providing an advertising bidding system in whichadvertisers can bid on advertising time slots or enhanced ads that canbe inserted into either prerecorded or live television programming.

The present invention may therefore comprise a process for auctioningadvertising time slots in a TV broadcast comprising: selecting apredetermined number of advertising time slots in the TV broadcast forauctioning; providing a description of the predetermined advertisingtime slots to interested advertisers; auctioning the selectedadvertising time slots to the interested advertisers in an auction;generating a TV advertisement for the selected advertising time slot fora TV advertiser that wins the auction; and transmitting the TVadvertisement in the selected advertising time slot.

The present invention may further comprise an embedded advertisingprocess for TV broadcasts comprising: identifying embedded advertisingopportunities in a TV program; providing the embedded advertisingopportunities to interested advertisers; auctioning the embeddedadvertising opportunities to the interested advertisers in an auctionsale; inserting an embedded advertisement in a TV program for a TVadvertiser that has won the auction sale; and transmitting the TVprogram with the embedded advertisement to viewers.

The present invention may further comprise an embedded TV advertisementgenerated by the process of: identifying embedded advertisingopportunities in a TV program; providing the embedded advertisingopportunities to interested advertisers; auctioning the embeddedadvertising opportunities to the interested advertisers in an auctionsale; inserting an embedded advertisement in a TV program for a TVadvertiser that has won the auction sale; and transmitting the TVprogram with the embedded advertisement to viewers.

The advantages of the present invention are that a high degree offlexibility can be provided by TV broadcasters by allowing embedded adsto be purchased and displayed in a very short time period. Thepurchasing process can be performed by an auctioning/bidding process.This allows TV broadcasters to increase revenues and allows advertisersto have more flexibility in advertising. Further, embedded advertisingtechniques allow for embedded ads to be displayed in accordance withregional demographics and even personal preferences of individualviewers. Objects can be inserted in the displayed video broadcast usingenhancements such as by using markup language overlay pages or by videocombination techniques. In addition, the effect of these ads can beevaluated to provide data for evaluating ad campaigns. For example;affinity data can be collected by advertisers and/or TV broadcasters toassist advertisers in the bidding process, such as disclosed in U.S.patent application Ser. No. 10/046,618, entitled “Collection of AffinityData from Television, Video or Similar Transmissions”, by Steven O.Markel, et al., filed Oct. 26, 2001, which is specifically incorporatedherein by reference for all that it discloses and teaches. Further, apredetermined number of standard advertising time slots can be includedin the “bid-for” advertising scheme which allows TV broadcasters to havea designated number of advertising slots which are pre-sold, so that apredetermined amount of revenue is guaranteed to the TV broadcasters.This allows TV broadcasters to eventually generate additional income byoffering some of the predetermined time slots as “bid-for” ads. Ofcourse, embedded advertisements can also be pre-sold or bid-for ads asdesired by the TV broadcaster. The embedded ads, in any event, providean additional source of revenue for the TV broadcaster.

In addition, preferences and demographic information that controls thevarious types of ads that are displayed to users can be provided toadvertisers prior to bidding on the embedded advertising. Demographicand preference information that controls the displayed advertising isdisclosed in U.S. patent application Ser. No. 10/688,067, filed Oct. 16,2003, entitled “Demographic/Preference Sniffer,” which is based uponU.S. provisional patent application Ser. No. 60/419,871, filed Oct. 18,2002, entitled “Demographic/Preference Sniffer,” both of which arespecifically incorporated herein by reference for all that they discloseand teach. Further, upgraded advertising, such as disclosed in U.S.patent application Ser. No. 10/688,075, filed Oct. 17, 2003, entitled“iChoose Video Advertising,” which is based upon U.S. provisionalapplication Ser. No. 60/420,110, filed Oct. 18, 2002, entitled “iChooseVideo Advertising,” is specifically incorporated herein by reference forall that it discloses and teaches. Information regarding advertisementschosen by viewers in accordance with the above identified applicationcan be used to further aid the TV advertisers in making bids on embeddedadvertising.

BRIEF DESCRIPTION OF THE DRAWINGS

In the drawings,

FIG. 1 is a schematic illustration of the overall system of the presentinvention;

FIG. 2 is a detailed block diagram of a set-top box, display, and remotecontrol device of the system of the present invention.

FIG. 3 is an illustration of an embodiment of the present inventionwherein four video signals may be combined into four composite videosignals.

FIG. 4 is an illustration of an embodiment of the present inventionwherein a main video image is combined with portions of a second videoimage to create five composite video signals.

FIG. 5 depicts another set top box embodiment of the present invention.

FIG. 6 depicts a sequence of steps employed to create a combined imageat a user's set top box.

FIG. 7 is an illustration of one implementation of the presentinvention.

FIG. 8 is an illustration of the manner in which interactive content maybe displayed.

FIG. 9 is a graphical representation of a database that provides a listof additional information associated with advertisers.

FIG. 10 is a graphical representation of a database that provides a listof attributes associated with advertisers.

FIGS. 11 and 12 are illustrations of scenes showing otherimplementations of the present invention.

FIGS. 13, 14, and 15 are illustrations of scenes showing otherimplementations of the present invention.

FIGS. 16 and 17 are illustrations of scenes showing otherimplementations of the present invention.

FIG. 18 is a schematic block diagram that generally illustrates oneembodiment of the present invention.

FIG. 19 is a schematic block diagram that generally illustrates anotherembodiment of the present invention.

FIG. 20 is a flow diagram that illustrates the steps that may beperformed in applying embedded ads into a video stream.

FIG. 21 is a flow diagram illustrating the steps that may be performedfor the installation of embedded ads by manual designation of availableadvertising space.

FIG. 22 is a flow diagram illustrating the steps that may be performedfor the installation of embedded ads by automatic designation ofavailable advertising space and searching for actual objects.

FIG. 23 is a flow diagram illustrating the steps that may be performedfor the installation of embedded ads by automatic designation ofavailable advertising space and searching for blank spaces.

FIG. 24 is a flow diagram illustrating the steps that may be performedfor the installation of embedded ads, using automatic methods, in amanner which combines the processes of looking for an actual object andblank space simultaneously.

FIG. 25 is a flow diagram illustrating the steps that may be performedfor the installation of embedded ads by using tags and markers todescribe advertising space.

FIG. 26 is a flow diagram illustrating a process of auctioning embeddedadvertising.

FIG. 27 is a flow diagram illustrating a process of biding on embeddedadvertising.

FIG. 28 is a flow diagram illustrating a process of modifying biddingstrategy based upon affinity data.

FIG. 29 is a flow diagram illustrating a process of bidding on liveevents.

FIG. 30 is a flow diagram illustrating a process of analyzingadvertising opportunities for live events.

FIG. 31 is a flow diagram illustrating a process for billing advertisersbased upon display time of embedded ads.

DETAILED DESCRIPTION OF THE INVENTION

FIGS. 1 through 6 illustrate one method of generating embeddedadvertising using video combination techniques. The combination ofsignals may occur in the set top box, or may occur in an upstreamsource. The embedded advertising can take the form of objects that areinserted into the TV programming using overlay pages that are generatedby markup language, by video combination techniques, or by any otherdesirable technique. Embedded advertising using overlay pages canreplace objects in the TV broadcast by negating the TV broadcast displayon a pixel by pixel basis so that objects can actually be eliminatedfrom the broadcast, such as disclosed in U.S. patent application Ser.No. 10/212,289, entitled “Post Production Visual Alterations,” by ThomasLemmons, filed on Aug. 2, 2002, which is specifically incorporatedherein by reference for all that it discloses and teaches.Alternatively, video combination techniques can be used to embed ads bycombining video images at the set top box level, such as disclosed inU.S. patent application Ser. No. 10/103,545, entitled “Video Combiner,”Thomas Lemmons, et al, filed Mar. 20, 2002, which is specificallyincorporated herein by reference for all that it discloses and teaches.Inserted objects can track with the background of the display andconform to objects in the display so that the inserted objects appear asobjects that are actually contained within the TV broadcast such asdisclosed in U.S. patent application Ser. No. 10/357,155, entitled“Tracking Moving Objects on Video with Interactive Access Points”, filedJan. 31, 2003, by Steven O. Markel which is specifically incorporatedherein by reference for all that it discloses and teaches. Embeddedadvertising provides a high degree of flexibility for auctioning ads.This high degree of flexibility allows TV broadcasters to offer embeddedadvertising which can be bid upon by TV advertisers for both prerecordedand live programming.

FIG. 1 illustrates the interconnections of the various components thatmay be used to deliver a composite video signal to individual viewers.Video sources 100 and 126 send video signals 102 and 126 through adistribution network 104 to viewer's locations 111. Additionally,multiple interactive video servers 106 and 116 send video, HTML, andother attachments 108. The multiple feeds 110 are sent to several settop boxes 112, 118, and 122 connected to televisions 114, 120, and 124,respectively. The set top boxes 112 and 118 may be interactive set topboxes and set top box 122 may not have interactive features.

The video sources 100 and 126 and interactive video servers 106 and 116may be attached to a conventional cable television head-end, a satellitedistribution center, or other centralized distribution point for videosignals. The distribution network 104 may comprise a cable televisionnetwork, satellite television network, Internet video distributionnetwork, or any other network capable of distributing video data.

The interactive set top boxes 112 and 118 may communicate to theinteractive video servers 106 and 108 though the video distributionnetwork 104 if the video distribution network supports two-waycommunication, such as with cable modems. Additionally, communicationmay be through other upstream communication networks 130. Such upstreamnetworks may include a dial up modem, direct Internet connection, orother communication network that allows communication separate from thevideo distribution network 104.

Although FIG. 1 illustrates the use of interactive set-top boxes 112 and118, the present invention can be implemented without an interactiveconnection with an interactive video server, such as interactive videoservers 106 and 116. In that case, separate multiple video sources 100can provide multiple video feeds 110 to non-interactive set-top box 122at the viewer's locations 111. The difference between the interactiveset top boxes 112 and 118 and the non-interactive set top box 122 isthat the interactive set top boxes 112 and 118 incorporate thefunctionality to receive, format, and display interactive content andsend interactive requests to the interactive video servers 106 and 116.

The set top boxes 112, 118, and 122 may receive and decode two or morevideo feeds and combine the feeds to produce a composite video signalthat is displayed for the viewer. Such a composite video signal may bedifferent for each viewer, since the video signals may be combined inseveral different manners. The manner in which the signals are combinedis described in the presentation description. The presentationdescription may be provided through the interactive video servers 106and 116 or through another server 132. Server 132 may be a web server ora specialized data server.

As disclosed below, the set-top box includes multiple video decoders anda video controller that provides control signals for combining the videosignal that is displayed on the display 114. In accordance withcurrently available technology, the interactive set-top box 112 canprovide requests to the interactive video server 106 to provide variousweb connections for display on the display 114. Multiple interactivevideo servers 116 can provide multiple signals to the viewer's locations111.

The set top boxes 112, 118, and 122 may be a separate box thatphysically rests on top of a viewer's television set, may beincorporated into the television electronics, may be functions performedby a programmable computer, or may take on any other form. As such, aset top box refers to any receiving apparatus capable of receiving videosignals and employing a presentation description as disclosed herein.

The manner in which the video signals are to be combined is defined inthe presentation description. The presentation description may be aseparate file provided by the server 132, the interactive video servers106 and 116, or may be embedded into one or more of the multiple feeds110. A plurality of presentation descriptions may be transmitted andprogram code operating in a set top box may select one or more of thepresentation descriptions based upon an identifier in the presentationdescription(s). This allows presentation descriptions to be selectedthat correspond to set top box requirements and/or viewer preferences orother information. Further, demographic information may be employed byupstream equipment to determine a presentation description version for aspecific set top box or group of set top boxes and an identifier of thepresentation description version(s) may then be sent to the set top boxor boxes. Presentation descriptions may also be accessed across anetwork, such as the Internet, that may employ upstream communication ona cable system or other networks. In a similar manner, a set top box mayaccess a presentation description across a network that corresponds toset top box requirements and/or viewer preferences or other information.And in a similar manner as described above, demographic information maybe employed by upstream equipment to determine a presentationdescription version for a specific set top box or group of set top boxesand an identifier of the presentation description version(s) may then besent to the set top box or boxes. The identifier may comprise a URL,filename, extension or other information that identifies thepresentation description. Further, a plurality of presentationdescriptions may be transferred to a set top box and a viewer may selectversions of the presentation description. Alternatively, softwareprogram operating in the set top box may generate the presentationdescription and such generation may also employ viewer preferences ordemographic information.

In some cases, the presentation description may be provided by theviewer directly into the set top box 112, 118, 122, or may be modifiedby the viewer. Such a presentation description may be viewer preferencesstored in the set top box and created using menus, buttons on a remote,a graphical viewer interface, or any combination of the above. Othermethods of creating a local presentation description may also be used.

The presentation description may take the form of a markup languagewherein the format, look and feel of a video image is controlled. Usingsuch a language, the manner in which two or more video images arecombined may be fully defined. The language may be similar to XML, HTMLor other graphical mark-up languages and allow certain video functionssuch as pixel by pixel replacement, rotation, translation, and deformingof portions of video images, the creation of text and other graphicalelements, overlaying and ghosting of one video image with another, colorkey replacement of one video image with another, and any other commandas may be contemplated. In contrast to hard-coded image placementchoices typical to picture-in-picture (PIP) display, the presentationdescription of the present invention is a “soft” description thatprovides freedom in the manner in which images are combined and that maybe easily created, changed, modified or updated. The presentation is notlimited to any specific format and may employ private or public formatsor a combination thereof. Further, the presentation description maycomprise a sequence of operations to be performed over a period of timeor over a number of frames. In other words, the presentation descriptionmay be dynamic. For example, a video image that is combined with anothervideo image may move across the screen, fade in or out, may be alteredin perspective from frame to frame, or may change in size.

Specific presentation descriptions may be created for each set top boxand tailored to each viewer. A general presentation description suitedto a plurality of set top boxes may be parsed, translated, interpreted,or otherwise altered to conform to the requirements of a specific settop box and/or to be tailored to correspond to a viewer demographic,preference, or other information. For example, advertisements may betargeted at selected groups of viewers or a viewer may have preferencesfor certain look and feel of a television program. In some instances,some presentation descriptions may be applied to large groups ofviewers.

The presentation descriptions may be transmitted from a server 132 toeach set top box through a backchannel 130 or other network connection,or may be embedded into one or more of the video signals sent to the settop box. Further, the presentation descriptions may be sent individuallyto each set top box based on the address of the specific set top box.Alternatively, a plurality of presentation descriptions may betransmitted and a set top box may select and store one of thepresentation descriptions based upon an identifier or other informationcontained in the presentation description. In some instances, the settop box may request a presentation description through the backchannel130 or through the video distribution network 104. At that point, aserver 132, interactive video server 106 or 116, or other source for apresentation description may send the requested presentation descriptionto the set top box.

Interactive content supplied by interactive video server 106 or 116 mayinclude the instructions for a set top box to request the presentationdescription from a server through a backchannel. A methodology fortransmitting and receiving this data is described in U.S. ProvisionalPatent Application entitled “Multicasting of Interactive Data Over ABack Channel”, filed Mar. 5, 2002 by Ian Zenoni, which is specificallyincorporated herein by reference for all it discloses and teaches.

The presentation description may contain the commands necessary forseveral combinations of video. In such a case, the local preferences ofthe viewer, stored in the set top box, may indicate which set ofcommands would be used to display the specific combination of videosuitable for that viewer. For example, in an advertisement campaign, apresentation description may include commands for combining severalvideo images for four different commercials for four different products.The viewer's preferences located inside the set top box may indicate apreference for the first commercial, thusly the commands required tocombine the video signals to produce the first commercial will beexecuted and the other three sets of commands will be ignored.

In operation, the device of FIG. 1 provides multiple video feeds 110 tothe viewer's locations 111. The multiple video feeds are combined byeach of the interactive set-top boxes 112, 118, 122 to generatecorrelated or composite video signals 115, 117, 119, respectively. Asdisclosed below, each of the interactive set-top boxes 112, 118, 122uses instructions provided by the video source 100, interactive videoservers 106, 116, a separate server 132, or viewer preferences stored atthe viewer's location to generate control signals to combine the signalsinto a correlated video signal. Additionally, presentation descriptioninformation provided by each of the interactive video servers 106, 116can provide layout descriptions for displaying a video attachment. Thecorrelated video signal may overlay the various video feeds on a fullscreen basis, or on portions of the screen display. In any event, thevarious video feeds may interrelate to each other in some fashion suchthat the displayed signal is a correlated video signal with interrelatedparts provided by each of the separate video feeds.

FIG. 2 is a detailed schematic block diagram of an interactive set-topbox together with a display 202 and remote control device 204. As shownin FIG. 2, a multiple video feed signal 206 is supplied to theinteractive set-top box 200. The multiple video feed signal 206 thatincludes a video signal, HTML signals, video attachments, a presentationdescription, and other information is applied to a tuner/decoder 208.The tuner/decoder 208 extracts each of the different signals such as avideo MPEG signal 210, an interactive video feed 212, another video orinteractive video feed 214, and the presentation description information216.

The presentation description information 216 is the informationnecessary for the video combiner 232 to combine the various portions ofmultiple video signals to form a composite video image. The presentationdescription information 216 can take many forms, such as an ATVEFtrigger or a markup language description using HTML or a similar format.Such information may be transmitted in a vertical blanking encodedsignal that includes instructions as to the manner in which to combinethe various video signals. For example, the presentation description maybe encoded in the vertical blanking interval (VBI) of stream 210. Thepresentation description may also include Internet addresses forconnecting to enhanced video web sites. The presentation descriptioninformation 216 may include specialized commands applicable tospecialized set top boxes, or may contain generic commands that areapplicable to a wide range of set top boxes. References made herein tothe ATVEF specification are made for illustrative purposes only, andsuch references should not be construed as an endorsement, in anymanner, of the ATVEF specification.

The presentation description information 216 may be a program that isembedded into one or more of the video signals in the multiple feed 206.In some cases, the presentation description information 216 may be sentto the set top box in a separate channel or communication format that isunrelated to the video signals being used to form the composite videoimage. For example, the presentation description information 216 maycome through a direct internet connection made through a cable modem, adial up internet access, a specialized data channel carried in themultiple feed 206, or any other communication method.

As also shown in FIG. 2, the video signal 210 is applied to a videodecoder 220 to decode the video signal and apply the digital videosignal to video RAM 222 for temporary storage. The video signal 210 maybe in the MPEG standard, wherein predictive and intracoded framescomprise the video signal. Other video standards may be used for thestorage and transmission of the video signal 210 while maintainingwithin the spirit and intent of the present invention. Similarly, videodecoder 224 receives the interactive video feed 212 that may comprise avideo attachment from an interactive web page. The video decoder 224decodes the video signal and applies it to a video RAM 226. Videodecoder 228 is connected to video RAM 230 and operates in the samefashion. The video decoders 220, 224, 228 may also perform decompressionfunctions to decompress MPEG or other compressed video signals. Each ofthe video signals from video RAMs 222, 226, 230 is applied to a videocombiner 232. Video combiner 232 may comprise a multiplexer or otherdevice for combining the video signals. The video combiner 232 operatesunder the control of control signals 234 that are generated by the videocontroller 218. In some embodiments of the present invention, ahigh-speed video decoder may process more than one video feed and thefunctions depicted for video decoders 220, 224, 228 and RAMs 222, 226,230 may be implemented in fewer components. Video combiner 232 mayinclude arithmetic and logical processing functions.

The video controller 218 receives the presentation descriptioninstructions 216 and generates the control signals 234 to control thevideo combiner 232. The control signals may include many commands tomerge one video image with another. Such commands may include directoverlay of one image with another, pixel by pixel replacement, colorkeyed replacement, the translation, rotation, or other movement of asection of video, ghosting of one image over another, or any othermanipulation of one image and combination with another as one mightdesire. For example, the presentation description instructions 216 mayindicate that the video signal 210 be displayed on full screen while theinteractive video feed 212 only be displayed on the top third portion ofthe screen.

The presentation description instructions 216 also instruct the videocontroller 218 as to how to display the pixel information. For example,the control signals 234 generated by the video controller 218 mayreplace the background video pixels of video 210 in the areas where theinteractive video feed 212 is applied on the top portion of the display.The presentation description instructions 216 may set limits as toreplacement of pixels based on color, intensity, or other factors.Pixels can also be displayed based upon the combined output of each ofthe video signals at any particular pixel location to provide a trulycombined output signal. Of course, any desired type of combination ofthe video signals can be obtained, as desired, to produce the combinedvideo signal 236 at the output of the video combiner 232. Also, anynumber of video signals can be combined by the video combiner 232 asillustrated in FIG. 2. It is only necessary that a presentationdescription 216 be provided so that the video controller 218 cangenerate the control signals 234 that instruct the video combiner 232 toproperly combine the various video signals.

The presentation description instructions 216 may include instructionssent from a server directly to the set top box 200 or the presentationdescription instructions 216 may be settable by the viewer. For example,if an advertisement were to be shown to a specific geographical area,such as to the viewers in a certain zip code, a set of presentationdescription instructions 216 may be embedded into the advertisementvideo instructing the set top box 200 to combine the video in a certainmanner.

In some embodiments, the viewer's preferences may be stored in the localpreferences 252 and used either alone or in conjunction with thepresentation description instructions 216. For example, the localpreferences may be to merge a certain preferred background with a newsshow. In another example, the viewer's local preferences may select froma list of several options presented in the presentation descriptioninformation 216. In such an example, the presentation descriptioninformation 216 may contain the instructions for several alternativepresentation schemes, one of which may be preferred by a viewer andcontained in the local preferences 252.

In some embodiments, the viewer's preferences may be stored in a centralserver. Such an embodiment may provide for the collection and analysisof statistics regarding viewer preferences. Further, customized andtargeted advertisements and programming preferences may be sent directlyto the viewer, based on their preferences analyzed on a central server.The server may have the capacity to download presentation descriptioninstructions 216 directly to the viewer's set top box. Such a downloadmay be pushed, wherein the server sends the presentation descriptioninstructions 216, or pulled, wherein the set top box requests thepresentation description instructions 216 from the server.

As also shown in FIG. 2, the combined video signal 236 is applied to aprimary rendering engine 238. The primary rendering engine 238 generatesthe correlated video signal 240. The primary rendering engine 238formats the digital combined video signal 236 to produce the correlatedvideo signal 240. If the display 202 is an analog display, the primaryrendering engine 238 also performs functions as a digital-to-analogconverter. If the display 202 is a high definition digital display, theprimary rendering engine 238 places the bits in the proper format in thecorrelated video signal 240 for display on the digital display.

FIG. 2 also discloses a remote control device 204 under the operation ofa viewer. The remote control device 204 operates in the standard fashionin which remote control devices interact with interactive set-top boxes,such as interactive set-top box 200. The set-top box includes a receiver242 such as an infrared (IR) receiver that receives the signal 241 fromthe remote 204. The receiver 242 transforms the IR signal into anelectrical signal that is applied to an encoder 244. The encoder 244encodes the signal into the proper format for transmission as aninteractive signal over the digital video distribution network 104 (FIG.1). The signal is modulated by modulator 246 and up-converted byup-converter 248 to the proper frequency. The up-converted signal isthen applied to a directional coupler 250 for transmission on themultiple feed 206 to the digital video distribution network 104. Othermethods of interacting with an interactive set top box may be alsoemployed. For example, viewer input may come through a keyboard, mouse,joystick, or other pointing or selecting device. Further, other forms ofinput, including audio and video may be used. The example of the remotecontrol 204 is exemplary and not intended to limit the invention.

As also shown in FIG. 2, the tuner/decoder 208 may detect web addressinformation 215 that may be encoded in the video signal 102 (FIG. 1).This web address information may contain information as to one or moreweb sites that contain presentation descriptions that interrelates tothe video signal 102 and that can be used to provide the correlatedvideo signal 240. The decoder 208 detects the address information 215which may be encoded in any one of several different ways such as anATVEF trigger, as a tag in the vertical blanking interval (VBI), encodedin the back channel, embedded as a data PID (packet identifier) signalin a MPEG stream, or other encoding and transmitting method. Theinformation can also be encoded in streaming media in accordance withMicrosoft's ASF format. Encoding this information as an indicator ismore fully disclosed in U.S. patent application Ser. No. 10/076,950,filed Feb. 12, 2002 entitled “Video Tags and Markers,” which isspecifically incorporated herein by reference for all that it disclosesand teaches. The manner in which the tuner/decoder 208 can extract theone or more web addresses 215 is more fully disclosed in the abovereferenced patent application. In any event, the address information 215is applied to the encoder 244 and is encoded for transmission throughthe digital video distribution network 104 to an interactive videoserver. The signal is modulated by modulator 246 and up-converted byup-converter 248 for transmission to the directional coupler 250 overthe cable. In this fashion, video feeds can automatically be provided bythe video source 100 via the video signal 102.

The web address information that is provided can be selected, asreferenced above, by the viewer activating the remote control device204. The remote control device 204 can comprise a personalized remote,such as disclosed in U.S. patent Ser. No. 09/941,148, filed Aug. 27,2001 entitled “Personalized Remote Control,” which is specificallyincorporated by reference for all that it discloses and teaches.Additionally, interactivity using the remote 204 can be provided inaccordance with U.S. patent application Ser. No. 10/041,881, filed Oct.24, 2001 entitled “Creating On-Content Enhancements,” which isspecifically incorporated herein by reference for all that it disclosesand teaches. In other words, the remote 204 can be used to access “hotspots” on any one of the interactive video feeds to provide furtherinteractivity, such as the ability to order products and services, andother uses of the “hot spots” as disclosed in the above referencedpatent application. Preference data can also be provided in an automatedfashion based upon viewer preferences that have been learned by thesystem or are selected in a manual fashion using the remote controldevice in accordance with U.S. patent application Ser. No. 09/933,928,filed Aug. 21, 2001, entitled “iSelect Video” and U.S. patentapplication Ser. No. 10/080,996, filed Feb. 20, 2002 entitled “ContentBased Video Selection,” both of which are specifically incorporated byreference for all that they disclose and teach. In this fashion,automated or manually selected preferences can be provided to generatethe correlated video signal 240.

FIG. 3 illustrates an embodiment 300 of the present invention whereinfour video signals, 302, 304, 306, and 308, may be combined into fourcomposite video signals 310, 312, 314, and 316. The video signals 302and 304 represent advertisements for two different vehicles. Videosignal 302 shows an advertisement for a sedan model car, where videosignal 304 shows an advertisement for a minivan. The video signals 306and 308 are background images, where video signal 306 shows a backgroundfor a mountain scene and video signal 308 shows a background for anocean scene. The combination or composite of video signals 306 and 302yields signal 310, showing the sedan in front of a mountain scene.Similarly, the signals 312, 314, and 316 are composite video signals.

In the present embodiment, the selection of which composite image todisplay on a viewer's television may be made in part with a localpreference for the viewer and by the advertiser. For example, theadvertiser may wish to show a mountain scene to those viewers fortunateenough to live in the mountain states. The local preferences may dictatewhich car advertisement is selected. In the example, the localpreferences may determine that the viewer is an elderly couple with nochildren at home and thus may prefer to see an advertisement for a sedanrather than a minivan.

The methodology for combining the various video streams in the presentembodiment may be color key replacement. Color key replacement is amethod of selecting pixels that have a specific color and location andreplacing those pixels with the pixels of the same location from anothervideo image. Color key replacement is a common technique used in theindustry for merging two video images.

FIG. 4 illustrates an embodiment 400 of the present invention wherein amain video image 402 is combined with portions of a second video image404. The second video image 404 comprises four small video images 406,408, 410, and 412. The small images may be inserted into the main videoimage 402 to produce several composite video images 414, 416, 418, 420,and 422.

In the embodiment 400, the main video image 402 comprises a border 424and a center advertisement 426. In this case, the border describestoday's special for Tom's Market. The special is the centeradvertisement 426, which is shrimp. Other special items are shown in thesecond video image 404, such as fish 406, ham 408, soda 410, and steak412. The viewer preferences may dictate which composite video is shownto a specific viewer. For example, if the viewer were vegetarian,neither the ham 408 nor steak 412 advertisements would be appropriate.If the person had a religious preference that indicated that they wouldeat fish on a specific day of the week, for example, the fish special406 may be offered. If the viewer's preferences indicated that theviewer had purchased soda from the advertised store in the past, thesoda advertisement 410 may be shown. In cases where no preference isshown, a random selection may be made by the set top box, a defaultadvertisement, or other method for selecting an advertisement may beused.

Hence, the present invention provides a system in which a correlated orcomposite video signal can be generated at the viewer location. Anadvantage of such a system is that multiple video feeds can be providedand combined as desired at the viewer's location. This eliminates theneed for generating separate combined video signals at a productionlevel and transmission of those separate combined video signals over atransmission link. For example, if ten separate video feeds are providedover the transmission link, a total of ten factorial combined signalscan be generated at the viewer's locations. This greatly reduces thenumber of signals that have to be transmitted over the transmissionlink.

Further, the present invention provides for interactivity in both anautomated, semi-automated, and manual manner by providing interactivevideo feeds to the viewer location. As such, greater flexibility can beprovided for generating a correlated video signal.

FIG. 5 depicts another set top box embodiment of the present invention.Set top box 500 comprises tuner/decoder 502, decoder 504, memory 506,processor 508, optional network interface 510, video output unit 512,and user interface 514. Tuner/decoder 502 receives a broadcast thatcomprises at least two video signals 501. In one embodiment of FIG. 5,tuner/decoder 502 is capable of tuning at least two independentfrequencies. In another embodiment of FIG. 5, tuner/decoder 502 decodesat least two video signals contained within a broadcast band, as mayoccur with QAM or QPSK transmission over analog television channel bandsor satellite bands. “Tuning” of Video signals may comprise identifyingpackets with predetermined PID (Packet Identifiers) values or a rangethereof and forwarding such packets to processor 508 or to decoder 504.For example, audio and/or video data packets 503 may be transferred todecoder 504 and control packets may be transferred to processor 508.Data packets may be discerned from control packets 507 through secondaryPIDs or through PID values in a predetermined range. Decoder 504processes packets received from tuner/decoder 502 and generates andstores image and/or audio information 505 in memory 506. Image and audioinformation 505 may comprise various information types common to DCTbased image compression methods, such as MPEG and motion JPEG, forexample, or common to other compression methods such as wavelets and thelike. Audio information may conform to MPEG or other formats such asthose developed by Dolby Laboratories and THX as are common to theatersand home entertainment systems. Decoder 504 may comprise one or moredecoder chips to provide sufficient processing capability to process twoor more video streams substantially simultaneously. Control packets 507provided to processor 508 may include presentation descriptioninformation. Presentation description information may also be accessedemploying network interface 510. Network interface 510 may be connectedto any type of network 511 that provides access to a presentationdescription including modems, cable modems, DSL modems, upstreamchannels in a set top box and the like. Network interface 510 may alsobe employed to provide user responses to interactive content to anassociated server or other equipment.

Processor 508 of FIG. 5 employs the presentation description to controlthe combination of the image and/or audio information stored in memory506. The combining process may employ processor 508, decoder 504, or acombination of processor 508 and decoder 504. The combined image and oraudio information, as created employing the presentation description, issupplied to video output unit 512 that produces and output signal for atelevision, monitor, or other type of display 515. The output signal 516may comprise composite video, S-video, RGB, or any other format. Userinterface 514 supports a remote control, mouse, keyboard or other inputdevice. User input 517 may serve to select versions of a presentationdescription or to modify a presentation description via user interface514.

FIG. 6 depicts a sequence of steps 600 employed to create a combinedimage at a user's set top box. At step 602 a plurality of video signalsare received. These signals may contain digitally encoded image andaudio data. At step 604 a presentation description is accessed. Thepresentation description may be part of a broadcast signal, or may beaccessed across a network. At step 606, at least two of the videosignals are decoded and image data and audio data (if present) for eachvideo signal is stored in a memory of the set top box. At step 608,portions of the video images and optionally portions of the audio dataare combined in accordance with the presentation description. Thecombination of video images and optionally audio data may producecombined data in the memory of the set top box, or such combination maybe performed “on the fly” wherein real-time combination is performed andthe output provided to step 610. For example, if a mask is employed toselect between portions of two images, non-sequential addressing of theset top box memory may be employed to access portions of each image in areal-time manner, eliminating the need to create a final display imagein set top box memory. At step 610 the combined image and optionallycombined audio are output to a presentation device such as a television,monitor, or other display device. Audio may be provided to thepresentation device or to an amplifier, stereo system, or other audioequipment.

The presentation description of the present invention provides adescription through which the method and manner in which images and/oraudio streams are combined may be easily be defined and controlled. Thepresentation description may specify the images to be combined, thescene locations at which images are combined, the type of operation oroperations to be performed to combine the images, and the start andduration of display of combined images. Further, the presentationdescription may include dynamic variables that control aspects ofdisplay such as movement, gradually changing perspective, and similartemporal or frame varying processes that provide image modification thatcorresponds to changes in scenes to which the image is applied.

Images to be combined may be processed prior to transmission or may beprocessed at a set top box prior to display or both. For example, animage that is combined with a scene as the scene is panned may beclipped to render the portion corresponding to the displayed image suchthat a single image may be employed for a plurality of video frames.

The combination of video images may comprise the process of replacingand/or combining a portion of a first video image with a second videoimage. The manner in which images are combined may employ any hardwareor software methods and may include bit-BLTs (bit block logictransfers), raster-ops, and any other logical or mathematical operationsincluding but not limited to maxima, minima, averages, gradients, andthe like. Such methods may also include determining an intensity orcolor of an area of a first image and applying the intensity or color toan area of a second image. A color or set of colors may be used tospecify which pixels of a first image are to be replaced by or to becombined with a portion of a second image. The presentation descriptionmay also comprise a mask that defines which areas of the first image areto be combined with or replaced by a second image. The mask may be asingle bit per pixel, as may be used to specify replacement, or maycomprise more than one bit per pixel wherein the plurality of bits foreach pixel may specify the manner in which the images are combined, suchas mix level or intensity, for example. The mask may be implemented aspart of a markup language page, such as HTML or XML, for example. Any ofthe processing methods disclosed herein may further include processesthat produce blurs to match focus or motion blur. Processing methods mayalso include processes to match “graininess” of a first image. Asmentioned above, images are not constrained in format type and are notlimited in methods of combination.

The combination of video signals may employ program code that is loadedinto a set top box and that serves to process or interpret apresentation description and that may provide processing routines usedto combine images and/or audio in a manner described by the presentationdescription. This program code may be termed image combination code andmay include executable code to support any of the aforementioned methodsof combination. Image combination code may be specific to each type ofset top box.

The combination of video signals may also comprise the combination ofassociated audio streams and may include mixing or replacement of audio.For example, an ocean background scene may include sounds such as birdsand surf crashing. As with video images, audio may be selected inresponse to viewer demographics or preferences. The presentationdescription may specify a mix level that varies in time or across aplurality of frames. Mixing of audio may also comprise processing audiosignals to provide multi-channel audio such as surround sound or otherencoded formats.

Embodiments of the present invention may be employed to add content toexisting video programs. The added content may take the form ofadditional description, humorous audio, text, or graphics, statistics,trivia, and the like. As previously disclosed, a video feed may be aninteractive feed such that the viewer may response to displayed imagesor sounds. Methods for rendering and receiving responses to interactiveelements may employ any methods and includes those disclosed inincorporated applications. Methods employed may also include thosedisclosed in U.S. application Ser. No. 10/403,317 filed Mar. 27, 2003 byThomas Lemmons entitled “Post Production Visual Enhancement Rendering”,which is a continuation-in-part of U.S. patent application Ser. No.10/212,289 filed Aug. 8, 2002 by Thomas Lemmons entitled “PostProduction Visual Alterations”, which is based upon U.S. ProvisionalPatent Application No. 60/309,714 filed Aug. 8, 2001 by Thomas Lemmonsentitled “Post Production Visual Alterations”, all of which arespecifically incorporated herein for all that they teach and disclose.As such, an interactive video feed that includes interactive contentcomprising a hotspot, button, or other interactive element, may becombined with another video feed and displayed, and a user response theinteractive area may be received and may be transferred over theInternet, upstream connection, or other network to an associated server.

FIGS. 7 through 25 illustrate, in more detail, the manner in whichenhancements, such as HTML overlay pages, can be used to create embeddedadvertising in TV programming.

FIG. 7 is an illustration of one implementation for providing embeddedadvertising using enhancements. Referring to FIG. 7, a scene from a TVbroadcast is displayed showing various items or objects in the scene.The video may be displayed on a display device 700, such as a televisionset, which may comprise any type of display such as a plasma display,flat panel display, CRT monitor, or any other display. The video mayalso be displayed on a computer monitor displaying embedded ads withinvideo streams over the Internet, or the video may be displayed on anyother type of display system/device including movies displayed at movietheaters. In the scene, a person 702 is shown as sitting on a chair 704at a table 706. The person 702 is seated in front of an object, such asa pizza box 708, from which the person is eating pizza. The object, orpizza box, may be automatically identified by some existing techniquesuch as pattern recognition software, as will be discussed in moredetail in FIG. 23. Alternatively, tags and markers, as disclosed in theabove referenced patent applications, may be supplied in the videostream that provides the object identification information (pizza box)and location of the object in the video stream. The system may thendetermine if a pizza advertiser has been listed as an embeddedadvertiser. That is, a list may be compiled that comprises advertiserswho wish to exhibit advertisements, in the form of embedded advertising,in a video presentation. The list of advertisers may also include theitem (attribute), which may be in the form of a product, idea, etc.,that the advertisers wish to promote. Such a list of advertisers andattributes may be stored in a database by the broadcaster. The list ofadvertisers and attributes is more fully disclosed with regard to thedescription of FIG. 9.

Referring again to FIG. 7, once an object, in this instance a pizza box708, is recognized, the advertiser list may be queried for a pizzaadvertiser. If a pizza advertiser has been listed as an embedded adadvertiser, an embedded ad, which may be in the form of a logo, text, orother form of advertisement for the advertiser, is graphically displayed(superimposed or overlaid) on the pizza box. The process of applying, orsuperimposing, the embedded on the pizza box is discussed in furtherdetail with regard to the description of FIG. 23. The embedded ad maycomprise any form of video enhancement, including video sequences, text,graphical images that may be in bitmap format or any other format, orother graphical representation. If the embedded ad is a “hot label,”(i.e., the embedded ad is an interactive enhancement), the viewer maythen click on the hot label to order a pizza from that advertised store.Details of this process are further discussed with respect to FIG. 23.

Embedded ads may comprise enhancements to the original video signal,such as improvements, augmentations or new additions. For example, acompany logo may be applied to a soda-can. The soda-can may alreadyexist within the original video signal, or the soda-can may be insertedinto the original signal. Alternatively, changes may comprise downgradesfrom the original video signal, such as degradations, reductions, ordeletions. For example, a portion of the video may be “erased” in orderto create a blank space, which may be used for applying embedded ads ordisplaying advertisements. Object insertion, in this fashion, can bedone using either video combination techniques, such as disclosed abovein FIGS. 1 through 7, or by using enhancements, such as markup languageoverlay pages, including interactive enhancements.

Embedded ads may be applied to any type of video content and videosignals such as live or taped a television broadcasts, animation, filmedvideo and the like or may be captured as a magnetic or optical mediafile, such as in a Digital Video/Versatile/Variable Disc (DVD)feature-length film. Embedded ads may also comprise informationalcontent such as markings. Content may be created by taping or filming,such as the filming of a live broadcast, a feature-length film, atelevision program, or other presentation. Content may also be createdusing animation. Cartoons on television or on feature-length films areone example of animation. Content creation may comprise computeranimation. Computer animation is a principal component of multimediapresentations including feature-length films, and many softwareapplications can be used to create animations that can be displayed onany display device, such as a computer monitor, television screen, ormovie screen.

Of course, embedded ads may be applied to any form of viewing material,including educational programs, multimedia presentations, orentertainment presentations. Such multimedia or entertainmentpresentations may include digitally enhanced or computer enhancedpresentations, animated presentations, which may include feature-lengthanimated films, animated programs that are broadcast over a televisionnetwork, DVD animated presentations, or other forms of animatedpresentations. Additionally, embedded ads may be applied to suchmultimedia or entertainment presentations as feature-length video filmsthat may be shown in movie theatres, perhaps in a digital format, overfiber-optic connections, through laser technology, or through othertechnology.

As mentioned above, techniques to facilitate supplementary content in avideo signal can also be accomplished by combining video signals. Such atechnique is disclosed in U.S. Provisional Application Ser. No.60/278,669, filed Mar. 20, 2001 entitled “DELIVERY OF INTERACTIVE VIDEOCONTENT USING FULL MOTION VIDEO PLANES”, now U.S. Non-Provisionalapplication Ser. No. 10/103,545, filed Mar. 20, 2002 entitled “VIDEOCOMBINER”, both of which are specifically incorporated herein byreference for all that they discloses and teach. In accordance with theabove referenced application, the system is capable of combining videosignals that incorporate embedded ads at the viewer's location. Forexample, multiple video feeds, some of which can include embedded ads,are provided to a viewer's set-top box together with instructions forcombining two or more video feeds. The video feeds can then be combinedin a set-top box or otherwise located at or near the viewer's locationto generate the combined or correlated video signal for display withembedded ads. As mentioned above, the video feeds can be combined by theTV broadcaster prior to transmission over the video distributionnetwork.

Additionally, one or more video feeds can comprise enhanced video thatis provided from an Internet connection. HTML-like scripting can be usedto indicate the layout of the enhanced video signal. Instructions can beprovided for replacement of particular segments comprising the embeddedads space, to make an interactive embedded ad.

Hot labels (interactive embedded ads) can appear unobtrusively during atelevision broadcast. As disclosed above, the embedded ad itself can bean interactive element which is placed on specific objects that relateto the advertising element, or in a blank portion of the screen. As alsodisclosed above, embedded ads may not be interactive, i.e., embedded adsmay exist simply as an advertisement or informational material that ispart of the video stream. Insertion of video content to produce combinedvideo streams, as well as insertion of interactive content can beaccomplished by using tags and markers inserted into the video stream asdisclosed in U.S. Non-Provisional application Ser. No. 10/076,950, filedFeb. 12, 2002, which is based upon U.S. Provisional Application Ser. No.60/268,350, filed Feb. 12, 2001 entitled “VIDEO TAGS AND MARKERS”, bothof which are specifically incorporated herein by reference for all thatthey disclose and teach.

Embedded ads that are interactive links can allow for superimposition ofinteractive embedded ad information on products. Embedded ads may appearas a visual enhancement, such as a logo or name, and thus anadvertisement, of any party, or advertiser, such as a local or nationalcompany, or simply informational. Embedded ads can also beindividualized, localized advertisements. An embodiment may employpattern recognition techniques that provide an automated method ofidentifying objects on which to place embedded ads that pertain to theobject. Embedded ads may be localized to a point where specificadvertisements can be superimposed from set top-boxes, so thatadvertising can be personalized down to the household or individual TVlevel.

Embedded ads may be applied to content by creating “embedded ad space,”or space within the content (which may be a video stream), that isavailable for application of an embedded ad. The designation of suchembedded ad space may be completed by manual or automatic methods. Theembedded ad or type of embedded ad to be superimposed may be chosenbased on “attributes,” as further explained with regard to thedescription of FIG. 9. A list of advertisers, along with theadvertisers' associated attributes, may be stored in a database.“Attributes” are a class of actual objects, or blank spaces that may beassociated with an advertiser or with types or classes of information.An embedded ad may be applied, to the embedded ad space within the videostream, according to the class of object or blank space identifiedwithin the video stream. The identified object or blank space (withinthe video stream) may be compared to objects, classes of objects orblank spaces listed as attributes within the aforementioned database.Once a matching actual object or blank space is found, an appropriateembedded ad is chosen and applied to the embedded ad space within thevideo stream.

Referring again to FIG. 7, the embedded ad 710 appears as a logo of alocal franchise called “Paul's Pizzeria.” The embedded ad may be anactual logo, name, or advertisement of a company that can be either alocal company or national chain and may exist as a non-obtrusiveadvertisement that appears to be an actual element of the originallyfilmed scene. The embedded ad 710 appears as a logo on the actual pizzabox, thus appearing completely naturally in the scene. The embedded ad710 may exist as a non-interactive enhancement that is part of the videosignal, and thus the embedded ad may simply exist as an advertisement,or the embedded ad may be an interactive element that has thefunctionality to be activated by the viewer, thus acting as anadvertisement and interactive tool.

Again referring to FIG. 7, if the embedded ad 710 is interactive, theembedded ad may be activated by the viewer 712. As previously described,the “viewer” is a person watching the video. Activation of the embeddedad 710 may occur in several ways, including toggling through a series ofembedded ads using control buttons on some form of a hand-held controldevice 716, such as a remote-control activating a cursor 718, using awired or wireless mouse, or by other means including but not by way oflimitation, light beams, such as IR beams that can be used as pointersto point at embedded ads. Such a system may operate in a fashion similarto a touch screen that identifies the light beam rather than a usersfinger. A cursor 718 may be an indicator, such as a spot, a flashingline, an arrow, a symbol, or any other indicator, that is displayed onthe TV screen 700. The cursor position may be controlled by another formof a hand-held control device, such as a wired mouse or a wireless mouse(not shown) in a similar fashion to that of a mouse being connected to acomputer and monitor. Once activated, the embedded ad may make availablea host of options, including bringing up an order form 720, which mayallow the viewer to begin the process of directly ordering a pizza. Theorder form 720 may appear on an unused portion of the TV screen 700,thereby causing no interruption of the current video presentation.

FIG. 8 is an illustration of the manner in which interactive content maybe displayed. Once activated, the embedded ad (as referred to in FIG. 7)may display information in an enhanced content window or border area801. The current video presentation shown on display device 800 mayremain viewable in the broadcast window 840. As shown in FIG. 8, thedisplayed video that includes the same objects that appeared in FIG. 7,including the table 826, pizza box 828, embedded ad 830, person 832, andchair 834, remain visible. Alternatively, the information brought up bythe embedded ad may be displayed on a completely separate page. Theembedded ad may bring up a graphical user interface (GUI) 802, a textbox, or some other presentation scheme that serves to presentinformation in some manner to the viewer. The GUI 802 may presentadditional information 804 about the company for which the embedded adhas been activated. Such additional information may be submitted by theadvertiser or by the author (person examining and/or editing the videosignal. The additional information 804 may be contained in a separatestorage device, such as a database. The database containing additionaladvertiser information is more fully disclosed with regard to thedescription of FIG. 9. Once an embedded ad is activated, thiscorresponding database can be accessed for the additional information.The separate database may be located in a central computer, or onmultiple computers, which may be a server at the head-end, aretransmission site, other site, or in the household of the viewer in aset-top box. The central computer is more fully disclosed with regard tothe description of FIGS. 17 and 18. The central computer inserts theembedded ad. Insertion of the embedded ad is discussed in further detailin FIG. 23.

Referring again to FIG. 8, the additional information 804 may includecompany profile 806, web address 808, coupons 810, or other information.The web address 808 may simply be an address that is non-interactive andis simply there for the viewer to take note of and to have theopportunity to visit that address at a later time, or the web addressmay act as a direct, interactive link. The interactive link 808 may linkthe viewer to the company home page, or the company website that allowsthe viewer to make a purchase 814. That is, the “order a pizza” website814 may be a special website designed specifically for viewerinteraction with embedded ads. In addition, if the advertiser is anational company, the additional information may include localdistributors or franchises 812 of the national company. The embedded admay provide a link to the local server to provide information regardingthe local store. For example, if Paul's Pizzeria is a national chain,the local broadcaster may superimpose the local franchise of Paul'sPizzeria at the server at the head-end, as illustrated in FIG. 17. Theviewer, when clicking on the embedded ad, will see their local Paul's'Pizzeria in the additional information display, and be able to order apizza directly from their local store.

Furthermore, the viewer may also have the choice of accessing links thathave nothing to do with the current video presentation. For example, ona different part of the graphical user interface 802, the viewer may beable to activate links 816 that facilitate access to the world-wide-web.For example, the viewer may be able to cast a vote 818 on whichcharacter the viewer liked the best during a particular televisionprogram such as the “Survivor” series. The viewer may also be able tocast a vote 820 for elections, read about their local politicalrepresentatives by clicking on an interactive link 822 and enteringtheir local information, or even read about current world events 824.

FIG. 9 is a graphical representation of a database that provides a listof additional information associated with advertisers, as listed indatabase 900. As previously mentioned with regard to the description ofFIG. 8, the additional information of an advertiser may be contained ina storage device, such as a database 900. As illustrated in FIG. 9, suchadditional information may be submitted as an entry 902. For example,one entry may be for the advertisers name, Paul's Pizzeria 901, withadditional associated information that may include company profile, webaddress, coupons, or other information. The additional information maybe submitted by the advertiser and/or by the author. Additionally, otheradvertisers may be stored in the same database along with similarinformation, as a separate entry 904.

FIG. 10 is a graphical representation of a database that provides a listof attributes associated with advertisers, as listed in database 1006.Database 1006 may also contain an embedded ad itself, or a link to anembedded ad. Referring to FIG. 10, in order to paint the embedded adonto a particular object or even onto a blank space, there must be adetermination of whether or not there is an appropriate embedded ad toapply. Thus, a list of advertisers is stored in a storage device, suchas a database 1006, along with associated attributes. “Attributes” aredefined as any actual object or categories of objects that areassociated with the advertiser, or “blank space,” which willsubsequently be explained. Attributes may be submitted by the advertiserand/or by the author. Referring to FIG. 10, an advertiser, as may beseen as one entry out of many, is stored in the database 1006. Forexample, the advertiser General Electric 1014 may be an entry 1008.General Electric may be accompanied by a list of attributes 1010. Suchattributes 1010 may include actual objects, such as “stove,”“refrigerator,” “toaster,” and other actual objects or categories suchas appliances, etc. Alternatively, advertisers may wish to have theopportunity to advertise on an available blank space; in this case, theadvertiser may submit “blank space” as one of their attributes. Forexample, the advertiser Joe's Appliances 1016, as seen in entry 1012,may elect to list “blank space” 1014 as an attribute, which would allowJoe's Appliances the opportunity to bid on a blank space in the videocontent as well as to apply an advertisement to an actual object. Blankspace may be further constrained to specific areas such as the dasherboards below the Colorado Avalanche hockey bench. Blank spaceadvertising is more fully disclosed with regard to the description ofFIGS. 11 and 12. Additionally, other advertisers may be stored in thesame database along with similar information, as other entries such as1018 and 1020. In addition, this database may store the file name ofand/or link to an embedded ad graphics file, which may be stored in aseparate location. Alternatively, the database may contain embedded adtext files 1026, 1028. These concepts are more fully disclosed withrespect to the description of FIGS. 13 and 14 The manner in which acompany is chosen as an advertiser using an embedded .ad is discussed ingreater detail in FIG. 24. If the system of the present invention findsa pizza advertiser in the list of advertisers, an embedded ad is paintedonto the identified pizza box, at which time the viewer may click on theembedded ad and order a pizza. The process of applying the embedded adis discussed in greater detail in FIG. 23.

FIGS. 11 and 12 are illustrations of scenes showing other embodiments.Embedded ads can be coded to change on a per-scene basis. As illustratedby FIG. 11, in scene one 1100 of a television show, a refrigerator 1102may be superimposed (overlaid) with an embedded ad 1104. In scene two1206, the camera angle is different, which changes the position of therefrigerator 1208. During this scene, the embedded ad 1210 remains onthe refrigerator. Furthermore, instead of the embedded ad appearing as anormal part of the scene as illustrated by embedded ads 1104 and 1210,an embedded ad 1212 may appear larger and more apparent, such as textwritten outside the boundaries of the area rug 1214, thus appearing tothe viewer as an obvious part of the scene.

FIGS. 13, 14 and 15 are illustrations of scenes showing otherembodiments. As shown in FIG. 13, a football game is displayed on a TVscreen 1300. The scene illustrates a football field 1302 and viewingstands with real banner ads 1304, 1306 and 1308 as typically found insports arenas. Embedded advertising is supplemented in this pre-existingvideo signal by superimposing embedded ads 1310, 1312, and 1314 on tothe playing field 1302. Thus, the final enhanced video signal appears tothe viewer as though the field embedded ads 1310, 1312, and 1314 areinherent to the setting and look as if they are a natural part of thescene.

In FIG. 14, a TV screen 1430 is shown wherein embedded ads 1434, 1436,and 1438 are superimposed over the top of the real banner ads thatexisted in the un-enhanced video signal. The enhanced video signalcompletely obscures the real banner ads present at the sports arena andappears to the viewer as though the superimposed embedded ads 1434,1436, and 1438 are inherent to the setting and are a natural part of thescene. In this way, local embedded advertising can supercede nationalembedded advertising for different regions of a nationally broadcastprogram. FIG. 14 also illustrates an informational embedded ad 1440placed on the football field 1432 denoting the end zone of the team onoffense.

In FIG. 15, a TV screen 1548 is shown wherein interactive player nameembedded ads 1550, 1552, 1554 and 1556 are superimposed over the helmetsof the football players in a game. In this instance, the enhanced videosignal is not viewed as inherent to the setting and the embedded ads maybe used as hot labels to bring up additional player information whenaccessed. This information can include personal, biographical orstatistical data or the like for the accessed player but requires theuser to click on the embedded ad.

FIGS. 16 and 17 are illustrations of scenes showing other embodiments.Each time that an embedded ad appears in a new scene, the embedded admay be a different advertisement. As shown in FIG. 16, scene one 1602,the person 1604 reaches into an open refrigerator 1606 for a soda can1608. The embedded ad 1610 is a “Coke” logo. In FIG. 17, scene two 1712,the same person 1714, while standing beside the closed refrigerator1716, drinks from the soda can 1718. The embedded ad 1720 is now a“Pepsi” logo. Applying embedded ads in this fashion may be done throughpattern recognition and localized superimposition of ads, as willsubsequently be explained.

FIG. 16 also illustrates the addition of an interactive tattoo embeddedad. In scene one 1602 the person 1604 is depicted with tattoo containingnationally recognized logo 1650 superimposed on the persons arm. In thisscene, the tattoo is overlaid with a transparent background to look likea natural part of the scene. Accessing this interactive hot label maylink the viewer to either the home page of the company depicted in thenationally recognized logo or to an address and phone number of a localtattoo shop.

FIG. 18 is a schematic block diagram that generally illustrates anotherembodiment. As shown in FIG. 18, an origination site 1800 may providecontent. Content comprises an un-enhanced video signal 1802. The contentmay be created by a content provider such as a TV network, TV studio, alive broadcast, a server providing code or data such as an ApplicationsService Provider, an Internet Service Provider, or any other type ofcontent provider that may be capable of sending content to viewers. Theorigination site 1800 may be capable of generating the content itself,or may simply provide content that is supplied from an outside source1804 in the form of an un-enhanced video signal 1806. The originationsite 1800 transmits the un-enhanced video signal 1802 to a server 1808.The address of the un-enhanced video signal can be provided by theadvertiser to the TV broadcaster.

Referring to FIG. 18, the server 1808 interacts with a central computer1810 and applies an embedded ad 1812 to the un-enhanced video signal1802. This embedded ad 1812 may comprise any form of video enhancement,including video sequences, text or graphical images and may be in anyformat, or other graphical representation such as bitmap or GIF. Theserver 1808 accesses on or more of the multiple data files a data files1822 contained in a central computer 1810. There may be one data filefor each advertiser, or one data file for each advertisement (embeddedad) and may comprise an embedded ad 1812 that exists as graphicalinformation within the data file. The data file 1822 may also compriseplacement and contour data 1814 which may be obtained by use of existingcomputer recognition, physical manipulation, or other techniques. Byusing placement and contour data, the embedded ad either may be appliedunobtrusively to the video signal, or may appear to be an obviousaddition to the original video signal 1802. Alternatively, the data file1822 may comprise secondary video streams that may be superimposed(overlaid) onto the original un-enhanced video signal 1802.

Referring again to FIG. 18, the central computer 1810 may also contain adatabase 1816, as referred to in the description of FIG. 9, which storeseach advertiser name and additional information. The central computer1810 may also contain another database 1818, as referred to in thedescription of FIG. 10, which stores each advertiser name, associatedattributes as well as a link 1022, 1024 to the embedded ad (asillustrated in FIG. 10) or an embedded ad text file 1026 and 1028. Eachentry for each advertiser in the database 1818 (as previouslyillustrated in FIG. 10) may be linked to the associated advertiser'sdata file 1822 though a link 1820. Again, the databases and the datafiles may be stored in a central computer or on multiple computers whichcan comprise a server located at a head-end or in a set-top box.

Referring to FIG. 18, the server 180′8 may incorporate existing patternrecognition software and techniques to provide one way in whichavailable advertising space may be identified. However, identificationof advertising space can be achieved by several methods as may befurther seen with respect to the description of FIG. 25. The patternrecognition software may communicate with the attribute database 1818,stored in the central computer 1810. As previously discussed, theattributes may be actual objects or “blank space” attributes. Thepattern recognition software may search the un enhanced video stream1802 these actual objects or blank spaces. Once a pattern is identified,the server searches the attribute database 1818 for a matchingattribute. In the event that more than one company has listed theidentified actual object or blank space as an attribute, bidding maystart to choose which advertiser will receive the embedded ad. Adetailed discussion of criteria, process and application of the embeddedad is described in further detail with respect to FIG. 23.

Referring again to FIG. 18, once the server 1808 has applied theembedded ad 1812 which the server accessed from either the data file1822, or the attribute and text-embedded ad database 1818, the outputfrom the server is an enhanced video signal 1824; i.e., a video signalwhich has one or more embedded ads applied to the video signal. Theenhanced video signal 1824 is transmitted via a link 1826 to a set-topbox 1828. The link 1826 may comprise a radio frequency link, satellitelink, cable connection, or any other type of communication. The set-topbox 1828 receives and transmits the enhanced video signal 1824 to adisplay device 1830, such as a computer monitor or TV screen.

FIG. 19 is a schematic block diagram that generally illustrates anotherembodiment. In FIG. 19, the embedded ad is applied by the set-top box,rather than the server. Content comprises an un-enhanced video signal1902. The content may be created by a content provider such as a TVnetwork, TV studio, a live broadcast, a server providing code or datasuch as an Applications Service Provider, an Internet Service Provider,or any other type of content provider that may be capable of sendingcontent to viewers. The origination site 1900 may be capable ofgenerating the content itself, or may simply provide content that issupplied from an outside source 1904 in the form of an un-enhanced videosignal 1906. The origination site 1900 transmits the un-enhanced videosignal 1902 via a link 1926, which again, may comprise a radio frequencylink, a satellite link, or other type of link, to the set-top box 1928.The set-top box 1928 accesses the central computer 1910 to apply one ormore embedded ads to the un-enhanced video signal 1902 to produce anenhanced video signal 1924 output. This enhanced video signal 1924 isthen sent from the set-top box 1928 to a display device 1930, such as acomputer monitor or TV screen.

FIG. 20 is a flow diagram that illustrates the steps performed inapplying embedded ads to a video stream 2000. As shown in FIG. 20,content is created at step 2002. As previously disclosed, the contentmay be created by a content provider such as a TV network, TV studio, alive broadcast, a server providing data such as an Applications ServiceProvider, an Internet Service Provider, or any other type of contentprovider that may be capable of sending content to viewers. The processis continued at step 2004 where available advertising space isdesignated by manual methods. Such manual methods may comprise having anauthor, i.e., a person examining the un-enhanced video signal, manuallydesignate places within the current un-enhanced video signal that areavailable for advertising. Alternatively, the process may proceed fromstep 2002, where the content is created, to step 2006, where thedesignation of available advertising space is automated. Automaticmethods of designating advertising space may include visual (pattern)recognition, or other techniques. The process proceeds to step 2008,where a list of the available advertising spaces is generated. This listmay be generated by a server that also contains a list of advertisersand their associated attributes. At step 2010, the available advertisingspace that has been recently designated is bid upon by the advertisers.At step 2012, embedded ads are placed in the video while the videocontent is being shown on a display device, such as a computer monitoror television screen.

Embedded ads may be representative of local or national companies andmay be superimposed on a per-household basis. This individualizedsuperimposition may be facilitated by utilizing affinity data, such asdisclosed in U.S. patent application Ser. No. 10/046,618, filed Oct. 26,2001 entitled “Collection of Affinity Data from Television, Video, orSimilar Transmissions,” which is based upon U.S. Provisional ApplicationSer. No. 60/243,506, filed Oct. 26, 2000 entitled “Collection ofAffinity Data from Television, Video, or Similar Transmission,” and alsodisclosed in U.S. Non-Provisional application Ser. No. 10/099,054, filedMar. 13, 2002 entitled “Affinity Marketing for Interactive MediaSystems,” which is based on U.S. Provisional Application Ser. No.60/275,295, filed Mar. 13, 2001 entitled “Affinity Marking forInteractive Television,” all of which are specifically incorporatedherein by reference for all that they discloses and teach. Embedded adsmay also be superimposed on a house-by-house basis by collecting datareceived from personalized remote controls, as disclosed in applicationSer. Nos. 60/227,916 and 09/941,148 referenced above. In accordance withthe above applications, affinity data provides information to a TVbroadcaster as to whether or not an audience is actually engaged in thecurrent TV video presentation, through use of a variety of sensors, andfeedback mechanisms as opposed to simply having the TV on and notwatching the TV. The personalized remote is used to collect dataregarding what TV shows are being watched by each different member of ahousehold. Together, the use of technology from affinity data andpersonalized remotes may allow for personal targeting of advertisements.In that regard, TV broadcasters may provide data to advertisers as tohow many households will receive their ad prior to starting the biddingprocess. This information may be stored with the attribute data andtransmitted to the advertiser when the TV broadcaster solicits bids.

FIG. 21 is a flow diagram illustrating the steps for carrying out theinstallation of embedded ads by manual designation of availableadvertising space 2100. The content, or video signal, is displayed instep 2101. In step 2102, the author, previously defined as the personexamining the video signal, examines the video signal. The processproceeds to step 2104, where the author manually designates availableadvertising space within the un-enhanced video signal. That is, theauthor looks for possible places existing in the original un-enhancedvideo stream onto which to place advertisements. Such possible placesmay include actual objects, onto which a particular logo of a companymay be applied, or blank spaces, onto which any advertiser desiring toplace their advertisement on a blank space may utilize. In addition, theauthor may plan the insertion of objects such as billboards, cans, cars,clothing, etc. In that regard, the existing objects in the program maybe changed. For example, the video signal may show an actress driving aChevy. The embedded advertising has the ability to change the Chevy to aFord. Once the author identifies an embedded advertising opportunity,the author designates the identified opportunity and placement spot asbeing available for an embedded advertisement. Proceeding to step 2106,placement and contour data for the identified object, an insertedobject, or blank space, which can act as the available advertisingspace, is generated. The placement/contour data may be obtained by useof existing computer recognition techniques, physical manipulation, orother techniques, and may be generated by a computer located at thehead-end, the set-top box, or other site. Utility of the contour datamay include making the embedded ad appear more naturally in the scene.As an example, if the object to which the embedded ad is applied isround, the embedded, as text or graphics or any visual enhancement, maybe wrapped around the object's physical shape so that the embeddedad-enhanced object appears to look natural. By generatingplacement/contour data, the available advertising space is designatedand comprises a boundary around the identified object.

Again referring to FIG. 21, in step 2108, the placement/contour data iscombined with the video signal. In step 2110, the combinedplacement/contour data and video signal are accessed by the computer.The location of the computer may be at the head-end (transmission site),retransmission site, the set top box (household), or some other site. Instep 2112, the embedded ad is applied to the un-enhanced video signal.As previously disclosed, the embedded ad may be applied by overlayingdata, which may contain the embedded ad itself along withplacement/contour data, onto the original un-enhanced video signal. Thedata may be in the form of a Hypertext Mark-up Language (HTML) overlaypage, which is mostly blank except for the embedded ad that the HTMLoverlay page contains. In this way, only the embedded ad is visible asthe video signal flows through the system. The server or set-top boxoverlays the HTML page onto the video signal, thus applying the embeddedad to selective areas of the video signal by using the coordinates givenby the placement/contour data. The embedded ad may be color-coded orpartially transparent such that the embedded ad appears to be part ofthe original broadcast to the viewer. Alternatively, the embedded ad maybe presented as an obvious addition to the original video signal, as waspreviously illustrated in FIG. 11.

The embedded ad may constitute graphical, image or text information andmay be in the form of an MPEG stream, a bitmap, a JPEG image, etc. Thegraphical, image or text information of the embedded ad may be storedlocally in the set-top box, at the head-end, at a distribution point ormay be accessed over the Internet or other network. As indicated above,an HTML overlay page may be used to combine the information of theembedded ad with the pre-existing video stream. However, any type ofcomputer-generated bitmap can be used to produce a screen that can becombined with the pre-existing video signal in one or more frames.

In addition, the computer-generated bitmap need not just overlay thepreexisting video image. For example, the combination of the video imageand the embedded ad image may employ arithmetic or logical combinationsof the bitmap information. Such combinations may employ raster opscommon to windowed graphical user interface environments such asMicrosoft Windows™. These arithmetic and logical combinations allow theembedded ad to replace or partially replace the pre-existing video imagewith the content of the embedded ad. Half toning and “see-through”embedded ads can also be produced using such techniques. The resultantvideo signal that is displayed to the viewer allows the embedded ad tothen appear as an integral part of the displayed image. Hence, theprocess of superimposing the embedded ad content on the pre-existingvideo content can be accomplished by simply overlaying a computergenerated bitmap on the pre-existing video content, which may alsocomprise bitmap information generated from an MPEG type of signal.Alternatively, the process of superimposing the embedded ad content onthe pre-existing video content can be accomplished by actually combiningthe computer generated bitmap containing the embedded ad content withthe pre-existing video content such as with arithmetic or logicalcombinations, as disclosed above. Further, commonly used color keyingtechniques, known in the television and computer arts and commonlyreferred to as “blue screen” and “chroma-key”, can also be employed. Forexample, the embedded ad content may include specific colors that havebeen designated to not replace the content of the pre-existing videocontent, or specific colors may be employed to define areas ofreplacement or combination. In this fashion, the “transparency” of theembedded ad content can be accomplished using color key techniques.

The process of selecting the embedded ad content can be accomplished invarious ways. For example, the embedded ad content may be accessed overa network connection. This network connection may be based uponpreferences stored in a set-top box or based upon a tag or marker thatis transmitted as part of the video content of the pre-existing videostream. In addition, the network connection may be an upstreamconnection to a server, such as a server located at the head-end or somedistribution point that has stored embedded ad content that can then bedownloaded to the set-top box either through the VBI or an out-of-bandchannel. In addition, the embedded ad content that is accessed in thisfashion may be based upon preferences that are stored either at thehead-end, an access point or at the set-top box. In addition, the bitmapinformation of the embedded ad content may be generated frominstructions that are downloaded from a network such as the variousnetwork connections described above. These instructions may compriseexecutable code that is capable of generating graphics or text at theset-top box level. Again, these can be based upon preferences.

While the foregoing description may employ references to a set-top box,it is specifically intended that any and all such description shallapply to any receiving unit operable to provide a similar function,including intelligent televisions, multimedia computers, satellitereceivers, etc., and also applies to portable systems that may employwireless networks to receive video transmission and that may employwireless networks to access embedded ad information.

FIG. 22 is a flow diagram illustrating the steps for carrying out theinstallation of embedded ads by automatic designation of availableadvertising space and searching for actual objects 2200. In previousembodiments of the invention, the designation of available advertisingspace was performed by manual methods. Alternatively, the designation ofopportunities for embedded advertising may be accomplishedautomatically, The following steps outline an automated process ofdesignating available advertising opportunities. In step 2201, thecurrent un-enhanced video signal is displayed. The process proceeds tostep 2202 where pattern recognition techniques may be applied to thecurrent un-enhanced video presentation. These pattern recognitiontechniques may be performed by a computer at a head-end (transmissionsite), retransmission site, the set top box (household), or some othersite. In step 2204, a query is made as to whether a pattern has beenrecognized. Recognized patterns may include actual objects. If a patternhas not been recognized, the process returns to step 2202 and continuesapplying the pattern recognition component to the current videopresentation. Returning to step 2204, if some sort of pattern isrecognized, the process proceeds to step 2206, where a query is made asto whether the recognized pattern is an actual object. If the recognizedpattern is not an actual object, the process returns to step 2202 andcontinues employing the pattern recognition component. After the queryat step 2206, if it is determined that the recognized pattern is anactual object, the process proceeds to step 2208 where placement andcontour data is generated, which may include coordinates of therecognized object: As previously disclosed, placement and contour datamay be generated by use of existing computer recognition techniques,physical manipulation, or other techniques. The placement/contour datamay be generated by a computer located at the head-end, the set-top box,or other site. By generating placement/contour data, the availableadvertising space is designated and comprises a boundary around theidentified object. Recognized objects can also be replaced automaticallyby other objects, as described above.

Referring again to FIG. 22, the placement/contour data is combined withthe video signal in step 2210. Thus, the output signal after step 2210is an enhanced video signal. The placement/contour data and videosignal, collectively referred to as the enhanced video signal, areaccessed by the computer in step 2212. As previously disclosed, thelocation of the computer may be at a head-end (transmission site),retransmission site, the set top box (household), or some other site. Instep 2214, the computer overlays a HTML page which is blank except forthe embedded ad, onto the video signal. Thus, one or more embedded adsare applied to the selective areas of the video signal by using thecoordinates given by the placement/contour data.

FIG. 23 is a flow diagram illustrating the steps for carrying out theinstallation of embedded ads by automatic designation of availableadvertising space and searching for blank spaces 2300. The principlesdiscussed in FIG. 23 are similar to the principles discussed in FIG. 22,with the pattern recognition component searching for a blank spaceinstead of an actual object. As previously disclosed in FIG. 22, thedesignation of available advertising space is completed by use ofautomated methods. The process begins at step 2301, where theun-enhanced video signal is displayed. The process continues at step2302, where pattern recognition techniques may be applied to theun-enhanced video signal. Again, the pattern recognition techniques maybe performed by a computer at a head-end (transmission site),retransmission site, the set top box (household), or some otherlocation. At step 2304, a query is made as to whether a pattern has beenrecognized. Recognized patterns may include blank spaces. If a patternhas not been recognized, the process returns to step 2304 and continuesapplying the pattern recognition component to the current videopresentation. Returning to step 2304, if a pattern is recognized, theprocess proceeds to step 2306, where a query is made as to whether therecognized pattern is a blank space. If the recognized pattern is not ablank space, the process returns to step 2302 and continues employingthe pattern recognition component. After the query at step 2306, if itis determined that the recognized pattern is a blank space, the processproceeds to step 1208 where placement and contour data is generated andmay include coordinates of the recognized object. As previouslydisclosed, placement and contour data may be generated by use ofexisting computer recognition techniques, physical manipulation, orother techniques by a computer at the head-end, set-top box, or othersite. By generating placement/contour data, the available advertisingspace is designated and comprises a boundary around the identifiedobject.

Referring again to FIG. 23, the placement/contour data is combined withthe video signal in step 2310. The resultant enhanced video signal isaccessed by the computer in step 2312. As previously disclosed, thelocation of the computer may be at the head-end (transmission site),retransmission site, the set top box (household), or some otherlocation. At step 2314 the computer overlays a HTML page, which is blankexcept for the embedded ad, onto the video signal. Thus, one or moreembedded ads are applied to the selective areas of the video signal byusing the coordinates given by the placement/contour data.

FIG. 24 is a flow diagram illustrating the steps for carrying out theinstallation of embedded ads, using automatic methods, in a manner thatcombines the processes of looking for an actual object while searchingfor a blank space simultaneously 2400. As shown in FIG. 24, the authorcreates a list of advertisers at step 2401. At step 2402, attributes areentered and assigned to respective advertisers. As previously defined,attributes are any actual object that is associated with the advertiser,or a blank space. Actual object attributes and blank space attributesare illustrated in FIGS. 11-15. As previously discussed in FIG. 8, eachadvertiser may be accompanied by a list of additional information.

Referring again to FIG. 24, at step 2404 a database record of eachcompany and their associated attributes, as illustrated in FIG. 10, iscreated. This record is saved to the database at step 2406. At step2408, the video signal, such as a television sitcom, is played. At step2410 a pattern recognition component is activated, which constantlysearches the video content to identify patterns, e.g., actual objects orbe blank spaces. At step 2412, a decision is made as to whether or not apattern has been recognized. If a pattern has been recognized, theprocess proceeds to step 2414. If a pattern has not been recognized, theprocess returns to step 2410 and continues searching the video signalfor additional patterns. If a pattern has been recognized at step 2414,a decision is made as to what type of pattern is recognized. If theidentified pattern is a blank space, the process proceeds to step 2416.At step 2416, placement and/or contour data is generated which mayinclude coordinates of the identified object. As previously disclosed,the placement and contour data may be generated by use of existingcomputer recognition techniques, physical manipulation, or othertechniques. By generating placement data, available advertising space isdesignated. At step 2418, the database is searched for an appropriateadvertisement. If no appropriate advertisement is found, the processreturns to step 2410 and continues searching the video signal. At step2420, the advertisers with matching attributes bid on the ad and thewinning bidder is chosen.

Referring back to FIGS. 13-15, the pattern recognition component mayrecognize a blank space, such as arena panels, open ice or field, etc.,onto which multiple advertisers may want to advertise (multipleadvertisers had listed “blank space” as one of their attributes). Anadvertiser may have chosen an attribute such as a hockey game, baseballgame, Friends episode, Seinfeld episode, situation comedy, prime timebroadcast, rerun, etc. to be placed in the bidding. This allows the TVadvertisers to be placed in the bidding process for areas of specificinterest and allows the TV broadcasters to target TV advertisers inplacing bids on topics, times, programs, etc. of interest to the TVadvertisers.

Referring again to FIG. 24, at step 2422, Extensible Mark-up Language(XML) code is created and superimposed (overlaid) onto the designatedblank space within the un-enhanced video signal. The XML code issuperimposed onto the un-enhanced video signal by use of theplacement/contour data previously acquired in step 2416. In the finalstep 2424, code is translated, via an Extensible Scripting Language(XSL) parser, from XML into code for Web TV set-top boxes, HTML, JavaScript (JS) (code for Liberate set-top boxes) and/or code for otherset-top boxes.

Referring again to FIG. 24, the process returns to step 2414. If thepattern recognized is an actual object, the process proceeds to step2426. At step 2426, placement and/or contour data is generated. Again,the placement and contour data may be generated by use of existingcomputer recognition techniques, physical manipulation, or othertechniques. At step 2428, the database is searched for a matchingattribute. If no matching attribute is found, the process returns tostep 2410 and the pattern recognition program continues searching thevideo stream. If a matching attribute is found, the process proceeds tostep 2430. At step 2430, the advertisers that have matching attributesare contacted to place a bid. The information regarding the embedded adis provided to the advertisers. The advertisers bid on the embedded adand the winning bid is chosen. If there are no advertisers with matchingattributes, the TV broadcaster may wish to categorize the opportunity asa blank space, which means that the object will be removed and a new oneinserted. This provides move opportunities to obtain bidders for theembedded advertising opportunities.

The process then proceeds to step 2432, where XML code is created andsuperimposed on the designated actual object or replaces the object. Inthe final step 2434, code is translated, via an XSL parser, from XMLinto code for Web TV set-top boxes, HTML, JS (code for Liberate set-topboxes) and/or code for other set-top boxes.

FIG. 25 is a flow diagram illustrating the steps performed for theinstallation of embedded ads by using tags and markers to describeadvertising space 2500. As mentioned previously, pattern recognitiontechniques may provide a way in which available advertising space may beidentified. However, identification of advertising space can be achievedby several methods. For example, available advertising space within thevideo stream may be defined by use of technology as disclosed inapplication serial numbers 60/268,350 and 10/076,950 referenced above.

Referring again to FIG. 25, content is created at step 2501. Proceedingto step 2502, available advertising space is designated. As discussed inFIGS. 21 and 22, the designation of available advertising space may beachieved by automatic or manual methods. If the available advertisingspace has been designated by automatic methods, it can then be checkedand edited by manual methods. The process proceeds to step 1404 wheretags are generated as disclosed in the above references. These tagsdescribe the advertising space that is designated in step 2502. Aspreviously disclosed, the advertising space may be an actual object or ablank space, and is referred to as an attribute. The tags define thelocation and perhaps contour of the identified object. That is, the tagsmay comprise a location or several defining locations, perhaps in x andy coordinates, and a contour of the object or blank space (attribute)identified. The tag may flag the attribute as being a blank space or anactual object. This placement and contour data may also be generated byuse of existing computer recognition techniques, physical manipulation,or other techniques. The process proceeds to step 2506, where the tag isplaced in the video stream.

Referring again to FIG. 25, the process proceeds to step 2508 wheremarkers are generated in a manner disclosed above. These markersindicate the position of a division between each video segment of thevideo stream. That is, the markers indicate the beginning and the end ofeach video segment within the video stream. The process proceeds to step2510, where the markers are inserted in the video stream. This can bedone by the content provider or marketer of the program.

At step 2512, a “moving location” tag may be superimposed into the videostream and that indicates the new location of an object on the screen ina manner such as disclosed in U.S. Provisional Application Ser. No.60/354,745, filed Feb. 6, 2002 entitled “TRACKING MOVING OBJECTS ONVIDEO WITH INTERACTIVE ACCESS POINTS”, which is specificallyincorporated herein by reference for all that it discloses and teaches.

For example, an identified object, which may be refrigerator, may moveslightly from one frame to the next. Such a tag would account for therefrigerator's new location and/or orientation, and this tag maycomprise new x and y coordinates and contour data of the refrigerator.The tags and markers may be sent during the video's vertical blankinginterval (VBI). Any known standard for transmitting enhanced data,including advanced television enhancement forum (ATVEF), may be used.References made herein to the ATVEF specification are made forillustrative purposes only, and such references should not be construedas an endorsement, in any manner, of the ATVEF specification.

Referring again to FIG. 25, the process proceeds to step 2514, where thevideo signal, which has been enhanced with tags and markers, istransmitted to a broadcaster or to the head-end where the tag is read.Alternatively, the broadcaster could read the tag, if the advertisementto be superimposed is a national advertisement. Referring again to step2514, the head-end reads the tag marking the location of an object orblank space and providing other information such as object or blankspace profile, size, type, or other information. The process thenproceeds to step 2516 where the head-end queries a list of advertisers,as referred to in FIG. 3. A query is made as to whether the listcontains an advertiser that has listed the identified object or blankspace as one of the advertiser's attributes. At step 2518, advertisersthat have matching attributes are selected for bidding and the owningbidder is selected. The embedded ad is consequently superimposed intothe video stream at head-end at step 2520. The display size of theembedded ad may be determined by the coordinate and contour dataprovided by the tag.

Various embodiments of the present invention therefore provide embeddedadvertising that can be bid upon by advertisers and implemented locally,or even on a household basis, which is the ultimate form of localizedadvertising. Localization of advertisements can be extremely effectiveespecially when combined with personal preference and demographic data.Furthermore, by adding embedded ads to video content in apost-production environment, local advertising can be provided in asimple and inexpensive manner. Targeted advertising in this manner, canbe very valuable to prospective advertisers, and advertisers will bewilling to bid high prices for such advertising. As such, revenue ofbroadcasters may be increased. Embedded ads may be unobtrusive,non-intrusive, and do not compromise size, color, or continuity of thevideo content. Embedded ads that are interactive may be activated byseveral methods, including clicking on the embedded ad with a hand-helddevice, or remote, moving a pointing device over the embedded ad, orother methods. Embedded ads can transfer information back to thehand-held device for display/interaction, perhaps by a wireless orinfrared link, send information back to a server for data aggregation,or perform any number of other actions. Further, a personalizedhand-held device can be used in accordance with the present inventionthat allows the system to provide personalized information and allowsseveral people to interact with the same program simultaneously. Suchprinciples are disclosed in U.S. Non-Provisional application Ser. No.09/941,148, filed Aug. 27, 2001 entitled “Personalized Remote Control,”which is based upon U.S. Provisional Application Ser. No. 60/227,916,filed Aug. 25, 2000 entitled “Personalized Remote Control,” both ofwhich are specifically incorporated herein by reference for all thatthey disclose and teach.

Embedded ads encourage impulse buying of products or services whichencourages advertisers to bid on embedded advertising. Embedded ads forproducts and services may be directly related to the content beingwatched, or may provide additional information for the purchase of itemsthat have no relevance to the video content. Embedded ads can tailorindividual one-on-one advertising, allow the viewer to vote, performother interactive responses, or facilitate access to the world-wide-web.Embedded ads can display information in a number of different ways, suchas on a relevant item, on a separate page, or on an unused portion ofthe video presentation. Traditional enhancements, such as a window ortext-box on the side or bottom of the television screen may appear oncethe embedded ad is activated. A special small icon, perhaps called a“Buy-Con™,” may appear in the corner of screen indicating to the viewerthe availability of embedded ads in the scene.

Embedded ads may be installed automatically by use of existing patternrecognition technology. Patterns that are identified by the software maybe actual objects that exist in the current video stream, or blankspaces that exist in the current video stream. Actual objects or blankspaces may be an appropriate place to superimpose an embedded ad. Thepatterns identified by the pattern recognition software are matched withthe list of attributes to identify an applicable embedded ad, and thepatterns are assigned placement and/or contour data. Theplacement/contour data is then used by a computer to apply the embeddedad in the current video stream. Objects can be replaced with otherobjects having the same or different attributes.

Alternatively, the installation of embedded ads may be achieved bymanually installing embedded ads into the programming content such asdisclosed in U.S. patent application Ser. No. 10/041,881, entitled“Creating On Content Enhancements,” which is based upon U.S. provisionalpatent application Ser. No. 60/243,107, entitled “Creating On ContentEnhancements,” referred to and incorporated above. Initially, manuallyinstalling advertising into the programming content may be done at thehead-end (transmission site), at a retransmission site, or in individualhouseholds according to their demographics. The principles of placingembedded ads on a household-to-household basis are disclosed in U.S.patent application Ser. No. 09/941,148, entitled “Personalized RemoteControl,” which is based upon U.S. provisional patent application Ser.No. 60/227,916, entitled “Personalized Remote Control,” referred to andincorporated above.

Further advantages of the various embodiments disclosed include theability to change embedded ads on a per-scene basis, and the ability toimplement embedded ads automatically or by manual methods. In addition,because embedded ads do not have to be interactive and can exist as partof the video signal. Televisions not enabled for interactivity are stillable to receive the embedded ads. Thus, the application of embedded adsmay not be restricted to video signals, but embedded ads may be appliedto any type of signal, and any type of medium.

The various embodiments disclosed provide a system and method thatallows post-production visual alterations in the form of bid-foradvertisements to be added to a video stream. In this manner, broadcastand basic cable channels may provide bid-for embedded advertising tolocal companies in a simple and inexpensive manner. Visual alterations,or embedded ads, may be superimposed into the video stream manually orautomatically. Embedded ads are an effective way to advertise, andembedded ads may be superimposed into the video stream on ahouse-by-house basis. Embedded ads may be personalized and localized.Embedded ads may be applied to any form of viewing material, within anytype of signal or medium. Embedded ads may appear unobtrusively orobviously within the video content, and embedded ads may be eitherinteractive or non-interactive. Embedded ads may change on a per-scenebasis.

In addition, preferences and demographic information that controls thevarious types of ads that are displayed to users can be provided toadvertisers prior to bidding on the embedded advertising. Demographicand preference information that controls the displayed advertising isdisclosed in U.S. patent application Ser. No. 10/688,067, filed Oct. 16,2003, entitled “Demographic/Preference Sniffer,” which is based uponU.S. provisional patent application Ser. No. 60/419,871, filed Oct. 18,2002, entitled “Demographic/Preference Sniffer,” which is specificallyincorporated herein by reference for all that it discloses and teaches.Further, upgraded advertising, such as disclosed in U.S. patentapplication Ser. No. 10/688,075, filed Oct. 17, 2003, entitled “iChooseVideo Advertising,” which is based upon U.S. provisional applicationSer. No. 60/420,110, filed Oct. 18, 2002, entitled “iChoose VideoAdvertising,” which is specifically incorporated herein by reference forall that it discloses and teaches. Information regarding demographicsand preferences that control the display of information can be used tofurther aid the TV advertisers in making bids on embedded advertising.

FIG. 26 with a flow diagram that illustrates the process of auctioningembedded advertising. At step 2602, the TV broadcaster generatesembedded advertising opportunities in the manner disclosed above. Atstep 2604, the TV broadcaster provides a list of embedded advertisingopportunities to advertisers and advertising specifications includingrequired ad information necessary to generate the embedded ad and otherinformation such as the expected duration of the ad, a description ofthe size of the ad, viewers who may be viewing the ad, targetedaudiences, appearance of the ad and bidding information. There aretargeted audiences, as indicated above, that are based upon eitherdemographic information or preference information of viewers. Thisinformation can be provided to the advertiser prior to bidding. The TVbroadcaster then accepts bids until the cutoff time at step 2606. Atstep 2608, the TV broadcaster selects the winning bidder. At step 2610,the TV broadcaster informs the bidders of the winning bidder and mayprovide additional opportunities at that point in time for placingadditional embedded ads to both the winning bidder and the losingbidders. At step 2614, the TV broadcaster obtains the required adinformation from the winning bidder prior to the ad deadline. The TVbroadcaster then generates the embedded ad at step 2616. At step 2618,the embedded ad is inserted into the TV broadcast in the mannerindicated above.

FIG. 27 is a flow diagram illustrating the steps 2700 that discloses theprocess performed by an advertiser in bidding on embedded advertising.At step 2702, the advertiser receives a list of embedded advertisingopportunities from the TV broadcaster. This list may be created, asdisclosed above, based upon attributes of the business of theadvertiser. For example, an advertiser may be in the business of sellingappliances, in which case, the advertiser may have listed “appliances”as an attribute. If there is an opportunity to place an embedded ad uponan appliance such as a refrigerator in a scene, the attribute“appliance” will be attached to the embedded advertising opportunity. Ofcourse, there may be many other examples of attributes which may be verybroad. As indicated above, the designation of an attribute of “reruns”which can be further limited to a type of programming, or a particularprogram, such as “situation comedy” or “Seinfeld” can be used as aseries of attributes so that an advertiser can obtain relevant embeddedadvertising information.

Referring again to FIG. 27, at step 2706, the advertiser reviews simplevideos, specifications for embedded ads, required ad information andbidding information. In this case, a simple video may be provided alongwith the embedded advertising opportunity which shows an ad embedded ina video clip so that the advertiser can have an idea as to the way inwhich the advertiser's ad may appear when the program is broadcast. Ofcourse, this process applies to prerecorded programming. However, withlive programming, such as a hockey game, video clips of previous gamesshowing embedded ads on the ice and on the boards can be provided as anexample of the manner in which an embedded ad may appear. Again, theadvertiser can determine the manner in which the advertiser's ad mayappear and may influence the amount that the advertiser may wish to bidon such ads.

At step 2708 of FIG. 27, the advertiser then determines if a bid will besubmitted. If not, the process proceeds to step 2710, and the advertiserreviews other embedded advertising opportunities. If the advertiserdetermines it wants to submit a bid, the process proceeds to step 2712to formulate a bidding strategy. Various bidding strategies are known inthe art of bidding and can be used by the advertiser. For example, anadvertiser may wish to submit a very high bid initially in an attempt todrive out other bidders. Of course, the advertiser may determine amaximum amount that the advertiser is willing to pay for the ad prior tosubmitting a first bid. In that regard, if affinity data is providedregarding the success of other embedded ads of the advertiser or thepopularity of the show, those factors may influence the maximum bidamount of the advertiser. At step 2714, the advertiser then submits abid. At step 2716, it is then determined whether the bid that has beensubmitted by the advertiser has won the auction. If it has not, theprocess proceeds to step 2722 where the advertiser may wish to reviewits bidding strategy. At step 2724, the advertiser then determineswhether it wishes to submit another bid. If not, the process proceeds tostep 2710 to review other advertising opportunities. If the advertiserdoes wish to submit another bid, the process returns to step 2714 tosubmit a bid. If the advertiser wins the bid at step 2716, the processproceeds to step 2718 where the advertiser provides the required adinformation to the TV broadcaster. The process then ends at step 2720.

FIG. 28 discloses the process 2800 of modifying bidding strategy basedupon affinity data. Affinity data is data that indicates viewers'responses to a program, an ad or any other information that isdisplayed. Affinity data can be collected by any desired methodincluding surveys, Nielsen rating data or in accordance with the processdescribed in U.S. patent application Ser. No. 10/046,618, entitled“Collection of Affinity Data from Television, Video or SimilarTransmissions”, filed Oct. 26, 2001, by Steven O. Markel, et al.

Referring again to FIG. 28, at step 2802, the advertiser determines thatit has won an advertising bid. At step 2804, the TV broadcastergenerates an enhanced ad. At step 2806, the enhanced ad is broadcast toviewers. At step 2808, affinity data is collected in accordance with anydesired method such as those described above. The affinity data is thengathered by the TV advertiser and analyzed at step 2810. At step 2812,the advertiser may wish to generate a new bidding strategy based uponthe affinity data. For example, if the advertiser is bidding on embeddedads for the Super Bowl and the game is a very close game in the fourthquarter, the advertiser may wish to bid greater amounts. Of course, thereverse is true, also. For example, if an advertiser is bidding in theSuper Bowl and the score is very lopsided in the fourth quarter, theadvertiser may wish to bid much lower amounts or withdraw a previous bidand pay a penalty fee.

Affinity data can also be used to determine the success of a particularad. For example, affinity data can be collected with regard to theresponse of viewers to an embedded ad that has been displayed previouslyin a program. For example, affinity data can rate a viewer's response toan ad which provides valuable information to the advertiser as towhether the advertiser wishes to bid higher or lower on subsequent adsduring an event such as a football game or hockey game.

Of course, the processes disclosed above are not limited to embedded adsonly. For example, TV broadcasters m. ay wish to designate certainadvertising time slots as ads which can be bid upon by TV advertisers.In this manner, a TV broadcaster may wish to maximize its revenue byhaving some time slots available for bidding while maintaining standardpre-purchased advertising time slots so that a guaranteed amount ofincome is provided. In addition, these time slots may have particularrelevance to what is occurring during the program. For example, a timeslot may be designated as a “bid-for” ad after a penalty is assessed ora play is being reviewed in a sporting event. An advertiser may wish topurchase that time slot and run a humorous advertisement showing areferee with thick glasses who has a hard time seeing. In addition, theaffinity data information can be very valuable for these types ofadvertisements to show viewer acceptance or rejection of theadvertisements. By collecting this data, TV advertisers can determine ifthey would like to run these ads on a more frequent basis and bid higherfor those particular time slots.

FIG. 29 discloses a bidding process 2900 that may be used for liveevents when the TV broadcaster is willing to extend the auction until atime that is very close to the time in which the embedded ad will bedisplayed. At step 2902, an advertiser receives the criteria foradvertising opportunities for the live event in the various ways asdisclosed above. At step 2904, the advertiser provides the requiredinformation for the embedded ads to the TV broadcaster. At step 2906,the TV broadcaster generates the embedded ads for each advertiser thathas submitted the required information and stores these embedded ads.Optionally, the TV broadcaster can provide a sample copy of the embeddedad to advertisers. At step 2908, the TV broadcaster sets the end ofauction time such that the end of auction time is close to the time fordisplay of the embedded ad. In this manner, bidding can occur during theprocess of the live event. For example, bidding may occur during actualplay of a game, such as the Stanley Cup finals, or the Super Bowl. Thecourse of events in the game can cause the bids to increase or decreaseduring the game as explained above. At step 2910, the bidders submitbids based upon the progress of the live event. At step 2912, a TVbroadcaster retrieves the embedded ad for the winning bidder. At step2914, the TV broadcaster displays the embedded ad.

FIG. 30 is a flow diagram illustrating the process 3000 of analyzingadvertising opportunities for live events. At step 3002, the TVbroadcaster can review various camera angles and various views of a liveevent or a similar event to attempt to determine embedded advertisingopportunities. For example, a TV broadcaster can view the camera anglesor previous video from a hockey game and determine what embeddedadvertising opportunities exist on the ice and on the boards of the icerink. As indicated above, even though markings may appear on the ice oron the boards, those markings can be removed using either markuplanguage overlay pages or video combination techniques, as disclosedabove. Hence, at step 3004, the potential locations for embedding adsare determined by the TV broadcaster. Advertisers may also assist the TVbroadcaster in that evaluation. At step 3006, the TV broadcastercollects all of the required ad information from interested advertisers.At step 3008, the various embedded ads, i.e., objects are generated forinsertion in the live program. At step 3010, the TV broadcaster can thenprovide a sample display of the embedded ads such as an inserted object.For example, during a broadcast of a live football game, an insertedobject may comprise an airplane carrying a banner or a banner that isinserted into the crowd in the stands. Other embedded objects, asdisclosed above, can comprise ads that are placed on the playersthemselves, such as tattoos or labels. At step 3012, the TV broadcasterobtains suggestions from the advertiser on modifications and additionalembedded advertising opportunities.

FIG. 31 discloses the process 3100 for billing advertisers based upondisplay time of embedded ads. At step 3102, the TV broadcaster embedsthe ad in the TV programming in the manner described above. At step3104, the embedding software counts the frames in which the full object,i.e., the embedded ad is displayed on the screen. At step 3106, thetotal count of frames for each embedded object is provided to thebilling software. At step 3108, the billing software generates a totaladvertising cost for the embedded ad based upon the total count offrames in which the embedded ad appears. Of course, a cap may exist thatlimits the advertisers exposure.

The present invention therefore provides various ways of allowing TVbroadcasters to auction both embedded ads and normal time slot ads.Auctioning of ads in this fashion allows TV broadcasters to potentiallyincrease advertising revenue for both prerecorded and live events.Advertisers, on the other hand, also benefit from the bidding process.For example, smaller companies may be able to advertise during largeevents such as the Super Bowl if the event has a smaller interestbecause of a lopsided score. Advertising can constitute targetedadvertising based upon preference and demographic data. In addition,affinity data, that evaluates the effect of the ad, can be used byadvertisers to maximize revenues and determine a bidding strategy.Embedded ads can be generated through markup language overlay pages orthrough combined video techniques. Various strategies for setting upembedded ads by TV broadcasters can be used including red zoneadvertising for football games, changing all of the embedded ads to asingle advertiser when the home team scores and providing time slotsthat are content related to a particular advertisement. Attributes of anembedded advertising opportunity can be used to select potentialadvertisers, which also allows potential advertisers to only reviewembedded ads that are of interest to the advertiser. Video combinationtechniques and markup language overlay pages can be used to eraseexisting ads in a scene and insert other ads, or insert other objects,into the scene. For example, if a table exists in a scene, a Coke can orbeer can be actually inserted into the scene and an existing item suchas an ashtray can be removed and a Coke can be inserted. Similarly, anobject such as a car can be erased and a different car can-be insertedin a scene to provide the embedded advertising opportunity. Variousadvertising opportunities may be provided based upon the occurrence ofan event in the program, such as the scoring of a goal or a touchdown.In that case, a trigger may be inserted in the program that triggers theinsertion of one or more embedded ads in the program. In this example, atrigger may be embedded in the video program which causes the highestbidder who has bid upon the go-ahead goal on the hockey game or thego-ahead score in a football game, etc. to have all the embedded adschanged to that particular advertiser's embedded ad. Various criteriacan be set up such as scores during the fourth quarter of a footballgame, scores during the last few minutes of a hockey or basketball game,etc. If advertisers wish to drop out of the bidding process, a backupfee for backing out of the process can be charged to the advertiser.Advertisers can also bid upon putting embedded ad in certain cameraangles. Hence, various methods are proposed for auctioning both embeddedadvertising and time slot advertising which provide many advantages toboth TV broadcasters and to TV advertisers.

The foregoing description of the invention has been presented forpurposes of illustration and description. It is not intended to beexhaustive or to limit the invention to the precise form disclosed, andother modifications and variations may be possible in light of the aboveteachings. The embodiment was chosen and described in order to bestexplain the principles of the invention and its practical application tothereby enable others skilled in the art to best utilize the inventionin various embodiments and various modifications as are suited to theparticular use contemplated. It is intended that the appended claims beconstrued to include other alternative embodiments of the inventionexcept insofar as limited by the prior art.

1-20. (canceled)
 21. A method comprising: presenting, on a displaydevice, a video having a series of interactive advertisements embeddedwithin scenes of the video; receiving a command to toggle through theseries of interactive advertisements; receiving a selection of aninteractive advertisement of the series of interactive advertisements totrigger activation of the interactive advertisement; and providing,using a hardware processor, further information corresponding to theselected interactive advertisement in response to the activation. 22.The method of claim 1, wherein the receiving the selection comprisesreceiving the selection from a hand-held device communicatively coupledto the display device.
 23. The method of claim 2, wherein the providingfurther information comprises providing the further informationcorresponding to the interactive advertisement to the hand-held devicefor display on the hand-held device.
 24. The method of claim 1, whereinthe interactive advertisement is embedded in a specific object within ascene of the video using placement and contour data for the specificobject.
 25. The method of claim 1, further comprising presenting an iconthat indicates an availability of the series of interactiveadvertisements.
 26. The method of claim 1, wherein the providing thefurther information comprises providing a form that allows a user tomake a purchase.
 27. The method of claim 1, further comprising providingthe further information in a content window separate from a broadcastwindow displaying the video on the display device.
 28. The method ofclaim 1, further comprising providing the further information in aborder area that is an appended portion to a broadcast window displayingthe video on the display device.
 29. The method of claim 1, furthercomprising providing the further information in an unused portion of thevideo.
 30. The method of claim 1, wherein the further informationcomprises a coupon corresponding to the interactive advertisement. 31.The method of claim 1, wherein the further information comprises a linkto access information regarding an entity corresponding to theinteractive advertisement.
 32. A method comprising: transmitting acommand, from a hand-held device to a computing device that ispresenting a video having a series of interactive advertisementsembedded within scenes of the video, to toggle through the series ofinteractive advertisements; and transmitting, using a hardwareprocessor, a selection of an interactive advertisement of the series ofinteractive advertisements to trigger activation of the interactiveadvertisement, the selection causing further information correspondingto the interactive advertisement to be presented.
 33. The method ofclaim 12, further comprising receiving the further informationcorresponding to the interactive advertisement, the further informationto be presented on the hand-held device.
 34. The method of claim 12,wherein the interactive advertisement is embedded in a specific objectwithin a scene of the video using placement and contour data for thespecific object.
 35. The method of claim 12, wherein the hand-helddevice is communicatively coupled to the computing device via a set-topbox.
 36. A system comprising: a computing device having a hardwareprocessor, the computing device to: present, on a display device, avideo having a series of interactive advertisements embedded withinscenes of the video; receive a command to toggle through the series ofinteractive advertisements; receive a selection of an interactiveadvertisement of the series of interactive advertisements to triggeractivation of the interactive advertisement; and provide, using thehardware processor, further information corresponding to the selectedinteractive advertisement in response to the activation.
 37. The systemof claim 16, wherein the computing device is further to present an iconthat indicates an availability of the series of interactiveadvertisements.
 38. The system of claim 16, wherein the interactiveadvertisement is embedded in a specific object within a scene of thevideo using placement and contour data for the specific object.
 39. Thesystem of claim 16, further comprising a hand-held devicecommunicatively coupled to the computing device that is presenting theseries of interactive advertisements, the hand-held device configuredto: transmit the command to toggle through the series of interactiveadvertisements; transmit the selection of the interactive advertisementto the computing device to trigger the activation of the interactiveadvertisement.
 40. The system of claim 19, wherein the hand-held deviceis further to receive the further information corresponding to theinteractive advertisement for display on the hand-held device.